San Francisco Housing Rental Analysis¶
In this assignment, you will perform basic analysis for the San Francisco Housing Market to allow potential real estate investors to choose rental investment properties.
In [1]:
# initial imports
import os
import pandas as pd
import matplotlib.pyplot as plt
import hvplot.pandas
import plotly.express as px
from pathlib import Path
from dotenv import load_dotenv
import panel as pn
%matplotlib inline
In [2]:
pn.extension('plotly')
In [ ]:
In [3]:
# Read the Mapbox API key
load_dotenv()
mapbox_token = os.getenv("MAPBOX")
Load Data¶
In [4]:
# Read the census data into a Pandas DataFrame
file_path = Path("Data/sfo_neighborhoods_census_data.csv")
sfo_data = pd.read_csv(file_path, index_col="year")
sfo_data.head()
Out[4]:
Housing Units Per Year¶
In this section, you will calculate the number of housing units per year and visualize the results as a bar chart using the Pandas plot function.
Hint: Use the Pandas groupby function
Optional challenge: Use the min, max, and std to scale the y limits of the chart.
In [5]:
# Calculate the mean number of housing units per year (hint: use groupby)
# YOUR CODE HERE!
In [6]:
sfo_data.groupby(level=0).housing_units.mean()
Out[6]:
In [7]:
# Use the Pandas plot function to plot the average housing units per year.
# Note: You will need to manually adjust the y limit of the chart using the min and max values from above.
# YOUR CODE HERE!
# Optional Challenge: Use the min, max, and std to scale the y limits of the chart
# YOUR CODE HERE!
#
In [8]:
fig_housing_units =sfo_data.groupby(level=0).housing_units.mean()
ax =fig_housing_units.plot(kind= 'bar', ylim= [370000,388500], title=' Housing Units in San Fransisco from 2010 to 2016')
ax.set_ylabel('Housing Units')
ax.set_xlabel('Year')
Out[8]:
Average Prices per Square Foot¶
In this section, you will calculate the average gross rent and average sales price for each year. Plot the results as a line chart.
Average Gross Rent in San Francisco Per Year¶
In [9]:
# Calculate the average gross rent and average sale price per square foot
# YOUR CODE HERE!
In [10]:
drop_column_sfo_data=sfo_data.drop('housing_units',1)
drop_column_sfo_data.groupby(level=0).mean()
Out[10]:
In [11]:
# Plot the Average Gross Rent per Year as a Line Chart
# YOUR CODE HERE!
In [12]:
average_gross_rent=drop_column_sfo_data.groupby(level=0).gross_rent.mean()
ax=average_gross_rent.plot(kind='line',title='Average Gross Rent in San Francisco')
ax.set_xlabel('Year')
ax.set_ylabel('Gross Rent')
Out[12]:
Average Sales Price per Year¶
In [13]:
# Plot the Average Sales Price per Year as a line chart
# YOUR CODE HERE!
In [14]:
average_sales_price= drop_column_sfo_data.groupby(level=0).sale_price_sqr_foot.mean()
ax = average_sales_price.plot(kind='line', title= 'Average Sales Price')
ax.set_xlabel('Year')
ax.set_ylabel('Avg. Sale Price')
Out[14]:
Average Prices by Neighborhood¶
In this section, you will use hvplot to create an interactive visulization of the Average Prices with a dropdown selector for the neighborhood.
Hint: It will be easier to create a new DataFrame from grouping the data and calculating the mean prices for each year and neighborhood
In [15]:
# Group by year and neighborhood and then create a new dataframe of the mean values
# YOUR CODE HERE!
In [16]:
new_sfo_data=sfo_data.reset_index()
new_sfo_data.groupby(['year', 'neighborhood'], as_index=False).mean().head(10).inplace=True
new_sfo_data.head(10)
Out[16]:
In [17]:
# Use hvplot to create an interactive line chart of the average price per sq ft.
# The plot should have a dropdown selector for the neighborhood
# YOUR CODE HERE!
In [18]:
new_sfo_data.hvplot.line(x= 'year', y="sale_price_sqr_foot", groupby= 'neighborhood')
Out[18]:
The Top 10 Most Expensive Neighborhoods¶
In this section, you will need to calculate the mean sale price for each neighborhood and then sort the values to obtain the top 10 most expensive neighborhoods on average. Plot the results as a bar chart.
In [19]:
# Getting the data from the top 10 expensive neighborhoods
# YOUR CODE HERE!
In [20]:
drop_year_new_sfo_data=new_sfo_data.drop('year',1)
most_expensive_neighborhood= drop_year_new_sfo_data.groupby(['neighborhood'], as_index=False).mean()
most_expensive_neighborhood.nlargest(10, columns='sale_price_sqr_foot')
Out[20]:
In [21]:
# Plotting the data from the top 10 expensive neighborhoods
# YOUR CODE HERE!
In [22]:
plot_m_e_n= most_expensive_neighborhood.nlargest(10, columns='sale_price_sqr_foot')
plot_m_e_n.hvplot.bar(x='neighborhood', y='sale_price_sqr_foot', rot=90 )
Out[22]:
Parallel Coordinates and Parallel Categories Analysis¶
In this section, you will use plotly express to create parallel coordinates and parallel categories visualizations so that investors can interactively filter and explore various factors related to the sales price of the neighborhoods.
Using the DataFrame of Average values per neighborhood (calculated above), create the following visualizations:
- Create a Parallel Coordinates Plot
- Create a Parallel Categories Plot
In [23]:
# Parallel Coordinates Plot
# YOUR CODE HERE!
In [ ]:
In [24]:
px.parallel_coordinates(plot_m_e_n, color='sale_price_sqr_foot')
In [25]:
# Parallel Categories Plot
# YOUR CODE HERE!
In [26]:
px.parallel_categories(plot_m_e_n, dimensions=['neighborhood', 'sale_price_sqr_foot', 'housing_units', 'gross_rent'], color='sale_price_sqr_foot', color_continuous_scale=px.colors.sequential.Inferno)
Neighborhood Map¶
In this section, you will read in neighboor location data and build an interactive map with the average prices per neighborhood. Use a scatter_mapbox from plotly express to create the visualization. Remember, you will need your mapbox api key for this.
Load Location Data¶
In [27]:
# Load neighborhoods coordinates data
file_path = Path("Data/neighborhoods_coordinates.csv")
df_neighborhood_locations = pd.read_csv(file_path)
df_neighborhood_locations.head()
Out[27]:
Data Preparation¶
You will need to join the location data with the mean prices per neighborhood
- Calculate the mean values for each neighborhood
- Join the average values with the neighborhood locations
In [28]:
# Calculate the mean values for each neighborhood
# YOUR CODE HERE!
In [29]:
mean_value_neighborhood=drop_year_new_sfo_data.groupby(['neighborhood'], as_index=False).mean()
mean_value_neighborhood.head(10)
Out[29]:
In [30]:
# Join the average values with the neighborhood locations
# YOUR CODE HERE!
In [31]:
average_value_per_neighborhood=pd.concat([df_neighborhood_locations, mean_value_neighborhood], axis=1).head(10)
average_value_per_neighborhood=average_value_per_neighborhood.drop('neighborhood', 1)
average_value_per_neighborhood.head(10)
Out[31]:
Mapbox Visualization¶
Plot the aveage values per neighborhood with a plotly express scatter_mapbox visualization.
In [32]:
# Create a scatter mapbox to analyze neighborhood info
# YOUR CODE HERE!
In [33]:
px.scatter_mapbox(average_value_per_neighborhood, lat='Lat', lon='Lon', size='sale_price_sqr_foot', color='gross_rent')
In [34]:
sfo_data.head()
Out[34]:
In [35]:
# Define Panel Visualization Functions
def housing_units_per_year():
fig_housing_units =sfo_data.groupby(level=0).housing_units.mean()
plot_housing_units = plt.figure()
ax =fig_housing_units.plot(kind= 'bar', ylim= [370000,388500], title=' Housing Units in San Fransisco from 2010 to 2016')
ax.set_ylabel('Housing Units')
ax.set_xlabel('Year')
plt.close(plot_housing_units)
return pn.pane.Matplotlib(plot_housing_units, tight=True)
# YOUR CODE HERE!
def average_gross_rent():
average_gross_rent=drop_column_sfo_data.groupby(level=0).gross_rent.mean()
fig_average_gross_rent = plt.figure()
ax=average_gross_rent.plot(kind='line',title='Average Gross Rent in San Francisco')
ax.set_xlabel('Year')
ax.set_ylabel('Gross Rent')
plt.close(fig_average_gross_rent)
return pn.pane.Matplotlib(fig_average_gross_rent, tight=True)
# YOUR CODE HERE!
def average_sales_price():
average_sales_price= drop_column_sfo_data.groupby(level=0).sale_price_sqr_foot.mean()
fig_average_sales_price = plt.figure()
ax = average_sales_price.plot(kind='line', title= 'Average Sales Price')
ax.set_xlabel('Year')
ax.set_ylabel('Avg. Sale Price')
plt.close(fig_average_sales_price)
return pn.pane.Matplotlib(fig_average_sales_price, tight=True)
# YOUR CODE HERE!
def average_price_by_neighborhood():
return new_sfo_data.hvplot.line(x= 'year', y="sale_price_sqr_foot", groupby= 'neighborhood')
# YOUR CODE HERE!
def top_most_expensive_neighborhoods():
plot_m_e_n= most_expensive_neighborhood.nlargest(10, columns='sale_price_sqr_foot')
return plot_m_e_n.hvplot.bar(x='neighborhood', y='sale_price_sqr_foot', rot=90 )
# YOUR CODE HERE!
def parallel_coordinates():
return px.parallel_coordinates(plot_m_e_n, color='sale_price_sqr_foot')
# YOUR CODE HERE!
def parallel_categories():
return px.parallel_categories(plot_m_e_n, dimensions=['neighborhood', 'sale_price_sqr_foot', 'housing_units', 'gross_rent'], color='sale_price_sqr_foot', color_continuous_scale=px.colors.sequential.Inferno)
# YOUR CODE HERE!
def neighborhood_map():
return px.scatter_mapbox(average_value_per_neighborhood, lat='Lat', lon='Lon', size='sale_price_sqr_foot', color='gross_rent')
# YOUR CODE HERE!
In [36]:
pn.extension()
In [40]:
row1 = pn.Row(average_price_by_neighborhood, average_gross_rent, average_sales_price)
row2 = pn.Row( parallel_coordinates)
row3 = pn.Row(top_most_expensive_neighborhoods)
row4 = pn.Row(parallel_categories)
summary = pn.pane.Markdown("""
# Summary Finding
### My Map
""")
In [42]:
title = pn.pane.Markdown("# Real Estate Analysis")
tabs = pn.Tabs (
("Averages", pn.Column(row1)),
("Top most Expensive and Parallel Graphs", pn.Column(row3,row2, row4)),
("Neighborhood Map", pn.Column(summary, neighborhood_map))
)
pn.Column(title, tabs)
Out[42]:
In [ ]:
In [ ]: